Towards practical automated human action recognition

University of Technology, Sydney. Faculty of Engineering and Information Technology.Modern video surveillance requires addressing high-level concepts such as humans' actions and activities. Automated human action recognition is an interesting research area, as well as one of the main trends in the automated video survei1lance industry. The typical goal of action recognition is that of labelling an image sequence (video) using one out of a set of action labels. In general, it requires the extraction of a feature set from the relevant video, fo1lowed by the classification of the extracted features. Despite the many approaches for feature set extraction and classification proposed to date, some barriers for practical action recognition sti11 exist. We argue that recognition accuracy, speed, robustness and the required hardware are the main factors to build a practical human action recognition system to be run on a typical PC for a real-time video surveillance application. For example, a computationally-heavy set of measurements may prevent practical implementation on common platforms. The main focus of this thesis is challenging the main difficulties and proposing solution. towards a practical action recognition system. The main outstanding difficulties that we have challenged in this thesis include 1) initialisation issues with model training: 2) feature sets of limited computational weight sui table for real-ti me application; 3) model robustness to outliers; and 4) pending issues with the standardisation of software interfaces. In the following, we provide a description of our contributions to the resolution of these issues. Amongst the different classification approaches for classifying action , graphical model such as the hidden Markov model (HMM) have been widely exploited by many researchers. Such models include observation probabilities which are generally modelled by mixtures of Gaussian components. When learning an HMM by way of Expectation-Maximisation (EM) algorithms, arbitrary choices must be made for their initial parameters. The initial choices have a major impact on the parameters at convergence and, in turn, on the recognition accuracy. This dependence forces us to repeat training with different initial parameters until satisfactory cross-validation accuracy is attained. Such a process is overall empirical and time consuming. We argue that one-off initialisation can offer a better trade-off between training time and accuracy, and as one of the main contributions of this thesis, we propose two methods for deterministic initialisation of the Gaussian components' centres. The first method is a time segmentation-based approach which divides each training sequence into the requested number of clusters (product of the number of HMM states and the number of Gaussian components in each state) in the time domain. Then, clusters' centres are averaged among all the training sequences to compute the initial centre for each Gaussian component. The second approach is a histogram-based approach which tries to initialise the components' centres with the more popular values among the training data in terms of density (similar to mode seeking approaches). The histogram-based approach is performed incrementally, considering each feature at a time. Either centre initialisation approach is followed by dispatching the resulting Gaussian components onto HMM states. The reference component dispatching method exploits the arbitrary order for dispatching. In contrast, we again propose two more intelligent methods based on the effort to put components with closer centres in the same state which can improve the co1Tect recognition rate. Experiments over three human action video datasets (Weizmann [1 ], MuHAVi [2] and Hollywood [3]) prove that our proposed deterministic initialisation methods are capable of achieving accuracy above the average of repeated random initialisations (about 1 per cent to 3 per cent in 6 random run experiment) and comparable to the best. At the same time, one-off deterministic initialisation can save the required training time substantially compared to repeated random initialisations, e.g. up to 83% in the case of 6 runs of random initialisation. The proposed methods are general as they naturally extend to other models where observation densities are conditioned on discrete latent variables, such as dynamic Bayesian networks (DBNs) and switching models . As another contribution, we propose a simple and computationally lightweight feature set, named sectorial extreme points, which requires only 1.6 ms per frame for extraction on a reference PC. We believe a lightweight feature set is more appropriate for the task of action recognition in real-time surveillance applications with the usual requirement of processing 25 frames per second (PAL video rate). The proposed feature set represents the coordinates of the extreme points in the contour of a subject's foreground mask. The various experiments prove the strength of the proposed feature set in terms of classification accuracy, compared to similar feature sets, such as the star skeleton [4] (by more than 3%) and the well-known projection histograms (up to 7%). Another main issue in density modelling of the extracted features is the outlier problem. The extraction of human features from videos is often inaccurate and prone to outliers. Such outliers can severely affect density modelling when the Gaussian distribution is used as the model since it is short-tailed and highly sensitive to outliers. Hence, outliers can affect the classification accuracy of the HMM-based action recognition approaches that exploit Gaussian distribution as the base component. In contrast, the Student' s t-distribution is more robust to outliers thanks to its longer tail and can be exploited for density modelling to improve the recognition rate in the presence of abnormal data. As another main contribution, we present an HMM which uses mixtures of t-distributions as observation probabilities and apply it for the recognition task. The conducted experiments over the Weizmann and MuHAVi datasets with various feature sets report a remarkable improvement of up to 9% in classification accuracy by using HMM with mixtures of t-distributions instead of mixture of Gaussians. Using our own proposed sectorial extreme points feature set, we have achieved the maximum possible classification accuracy (100%) over the Weizmann dataset. This achievement should be considered jointly with the fact that we have used a lightweight feature set. On a different ground, and from the implementation viewpoint, surveillance software for automated human action recognition requires portability over a variety of platforms, from servers to mobile devices. The current products mainly target low level video analysis tasks, e.g. video annotation, instead of higher level ones, such as action recognition. Therefore, we explore the potential of the MPEG-7 standard to provide a standard interface platform (through descriptors and architectures) for human action recognition from surveillance cameras. As the last contribution of this work, we present two novel MPEG-7 descriptors, one symbolic and the other feature-based, alongside two different architectures: the server-intensive which is more suitable for "thin" client devices , such as PDAs and the client-intensive that is more appropriate for ''thick" clients, such as desktops. We evaluate the proposed descriptors and architectures by way of a scenario analysis. We believe that through the four contributions of this thesis, human action recognition systems have become more practical. While some contributions are specific to generative models such as the HMM, other contributions are more general and can be exploited with other classification approaches. We acknowledge that the entire area of human action recognition is progressing at an enormous pace, and that other outstanding issues are being resolved by research groups world-wide. We hope that the reader will enjoy the content of this work

Robust density modelling using the student's t-distribution for human action recognition

The extraction of human features from videos is often inaccurate and prone to outliers. Such outliers can severely affect density modelling when the Gaussian distribution is used as the model since it is highly sensitive to outliers. The Gaussian distribution is also often used as base component of graphical models for recognising human actions in the videos (hidden Markov model and others) and the presence of outliers can significantly affect the recognition accuracy. In contrast, the Student's t-distribution is more robust to outliers and can be exploited to improve the recognition rate in the presence of abnormal data. In this paper, we present an HMM which uses mixtures of t-distributions as observation probabilities and show how experiments over two well-known datasets (Weizmann, MuHAVi) reported a remarkable improvement in classification accuracy. © 2011 IEEE

Human action recognition with MPEG-7 descriptors and architectures

Modern video surveillance requires addressing high-level concepts such as humans' actions and activities. In addition, surveillance applications need to be portable over a variety of platforms, from servers to mobile devices. In this paper, we explore the potential of the MPEG-7 standard to provide interfaces, descriptors, and architectures for human action recognition from surveillance cameras. Two novel MPEG-7 descriptors, symbolic and feature-based, are presented alongside two different architectures, server-intensive and client-intensive. The descriptors and architectures are evaluated in the paper by way of a scenario analysis

Histogram-based training initialisation of hidden Markov models for human action recognition

Human action recognition is often addressed by use of latent-state models such as the hidden Markov model and similar graphical models. As such models require Expectation-Maximisation training, arbitrary choices must be made for training initialisation, with major impact on the final recognition accuracy. In this paper, we propose a histogram-based deterministic initialisation and compare it with both random and a time-based deterministic initialisations. Experiments on a human action dataset show that the accuracy of the proposed method proved higher than that of the other tested methods. © 2010 IEEE

Deterministic initialization of hidden Markov models for human action recognition

Human action recognition is often approached in terms of probabilistic models such as the hidden Markov model or other graphical models. When learning such models by way of Expectation-Maximisation algorithms, arbitrary choices must be made for their initial parameters. Often, solutions for the selection of the initial parameters are based on random functions. However, in this paper, we argue that deterministic alternatives are preferable, and propose various methods. Experiments on a video dataset prove that the deterministic initialization is capable of achieving an accuracy that is comparable to or above the average from random initializations and suffers from no deviation thanks to its deterministic nature. The methods proposed naturally extend to be used with other graphical models such as dynamic Bayesian networks and conditional random fields. © 2009 IEEE

Poređenje gustine nasda nereis diversicolor u prirodnim i uzgojnim (anzali lagona) uslovima

Nereis diversicolor is regarded as a live food and is significantly effective in increasing stocks, enhancing tolerance in sturgeons and also increasing survival of sturgeon fry. Research conducted indicates that N. diversicolor is more abundant in May as compared to other months of the year. In February, only breeders of this species are found in the environment. N. diversicolor was found in four different weight classes from March 2009 to February 2010. They showed decrease in density with increase in temperature and reached the lowest numbers in February. In years 2009 and 2010, 200 sampling conducted monthly at the point where Caspian Sea mixes with the Anzali lagoon. Sampling was performed by Van Veen sampler with 400 cm2 cross section. Sediments were washed through a sieve with 0.5 mm mesh size. The residue along with Nereis was transferred to a dish, worms separated and transferred to lab. The density of worms per m2 was calculated according to the density formula. Two experiments were conducted to determine the best stocking density for the culture of N. diversicolor. Stocking density of 381-6350 worms m-2 were used in 6 trials initially and best growth was recorded in trial with density of 381 worms m-2. In the second experiment using 381-3175 worms m-2 in 7 trials, trial with 381 worms m-2 again showed better growth as compared to other trials. These results obtained were almost similar to those obtained for N. diversicolor is in its natural environment (447 worms m-2)

Accuracy of Visual inspection with acetic acid (VIA) for early detection of cervical dysplasia in Tehran, Iran

Objective: To evaluate the accuracy of visual inspection with 5 acetic acid (VIA) when used to detect cervical cancer and its precursors. Methods: The study population included women attended Family Planning and Gynecological Clinic in Bagher Abad Health Center and Mirza Koochak Khan Hospital for regular cervical screening tests. After obtaining informed consent from each woman, VIA was performed. One hundred with a positive VIA test and 100 women with a negative VIA test were randomly selected for this study. Cytology and colposcopy examination were performed for all 200 cases and cervical biopsies were conducted for those individuals showing abnormal colposcopic findings. Results: Nine cases in VIA-positive group and two cases in VIA-negative group had an abnormal cytology. Ninety five women in the VIA-positive group and 25 in the VIA-negative group had abnormal colposcopic findings. From biopsy examination, 67 (71) of cases in the VIA-positive group and 3 (12) cases in the VIA-negative group had a final diagnosis of dysplasia. Among biopsied samples, only 7 cases of VIA-positive group showed abnormal result and the remaining were normal. Based on these results, VIA test sensitivity and specificity were 95.7 and 44.0 respectively, while they were 10 and 92 for cytology tests. Conclusions: The results of this study indicate that although VIA is a sensitive screening test for detection of cervical dysplasia, it can not be used by itself. Applying VIA along with Pap smears helps to detect a higher number of cases with cancer precursor lesions

Efekat tri različite hrane na rast i preživljavanje larvi persijske jesetre (acipenser persicus)

A feeding experiment was conducted to evaluate growth and survival of Persian sturgeon larvae fed with live food (Nereis diversicolor and Daphnia spp.) and artificial diet. Diets were included: Diet 1: Dafnia (Dafnia magna), Diet 2: Nereis diversicolor worm; Diet 3: Mix of Daphnia (Daphnia spp.) (50%) and Nereis diversicolor (50%), Diet 4: Mix of Nereis diversicolor (50%) and concentrate food (50%) and Nereis diversicolor (50%); Diet 5: Mix of Daphnia spp. (33.33%) and Nereis diversicolor (33.33%) and concentrate food (33.33%). Persian sturgeon larvae were distinctly transferred to 15 tanks and fed for 15 days. For each treatment, 60 larvae were stocked into tanks. The total length and body weight of the fish were determined once before initiation of the experiment and at the end of the experimental period to assess their growth performance. Water quality parameters were recorded two times a day. There was significant difference (P0.05) were not found between diets 2 and 3 and also between diets 4 and 5. The value for BWI, PBWI, GR, DGI, SGR was higher in larvae fed mix of Nereis diversicolor and Daphnia spp. (diet 3) (p<0.05) and there was significant difference between all of groups. Except diet 3 there was no significant difference in the CF of fish fed the survey diets

Malignant mixed mullerian tumor of the uterus associated with tamoxifen therapy in a patient with a history of breast cancer

Tamoxifen is the drug of choice in the treatment of breast cancer. Recent reports show an increased incidence of endometrial carcinoma in patients taking tamoxifen. In this article, we report a case of malignant mixed mullerian tumor after tamoxifen use. Copyright © 2006 by Razi Institute for Drug Research (RIDR)